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Abstract 


This paper presents an analysis of teachers’ ratings on distance learning undergraduate 
study programs: 7,156 students enrolled in traditional and 528 students enrolled in 
distance learning studies took part in the evaluation questionnaire, assessing 71 
teachers. The data were collected from the Moodle platform and from the Singidunum 
University information system, and then analysed with SPSS statistical software. The 
parameters considered as potentially affecting teacher ratings are: number of teachers 
engaged in a particular course, total number of courses in which the teacher is engaged, 
teacher’s gender and age, total number of the available resources, and so forth. The 
results imply that scores assigned to individual teachers are consistent in both 
traditional and distance learning programs. The average rate was perceived to be lower 
when there were several teachers in a single course; such an effect was enhanced in 
cases where there was a significant age discrepancy among them. The other factors 
considered did not show a significant association with teacher ratings. Students’ main 
remarks about the work of the teachers have been summarised at the end of this paper. 
Possible explanations and implications of the results are discussed and 
recommendations are given. 
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Introduction 


Distance learning is becoming increasingly recognised as a suitable and valuable 
educational experience (Davies, Howell, & Petrie, 2010). Many universities across the 
globe offer distance learning courses through the introduction of learning management 
systems that allow them to have both on- and off-campus students. Since recruitment 
and retention of students has a significant financial impact on today’s universities, there 
have been numerous studies on student satisfaction and academic achievements from 
distance learning programs (e.g., Endres, Chowdhury, Fiye, & Hurtubis, 2009; Eom, 
Wen, &Ashill, 2006). 

One of the main factors influencing student satisfaction is the quality of teaching. This is 
an especially important consideration for university managers and decision makers, 
who can organise distance learning in a way that allows them to provide ongoing 
guidance and improvement strategies for teaching staff. On the contrary, some other 
factors are not easily influenced and managed, such as student self-motivation, student 
learning style, required/elective courses ratio, and so forth. Rating teachers should be a 
valuable procedure for students as well, because it can lead to improvement of teaching 
quality, based on the stated opinions of the students (Marzano, 2012; Nargundkar & 
Shrikhande, 2012). The study of Taylor and Tyler (2012) strongly confirms the opinion 
that teachers develop skills and otherwise improve due to student evaluation. They 
found that teachers were more successful at improving student achievement during the 
school year when they were evaluated than in the years before the evaluation, and were 
even more successful in the years after the evaluation. However, some reports (e.g., 
Toch & Rothman, 2008) show that teacher evaluations have not helped in the formation 
of highly skilled teachers. We presume that teachers’ ratings could influence their skills 
development provided that their tenure depends on the rating, the rating is publicly 
announced (which puts additional pressure on the teacher), and there are some 
consequences for the teachers with lowest scores. Since there are no widely accepted 
standards of teacher performance, universities around the world mainly use student 
evaluation of teachers as the principal source of data on teachers’ performance and 
quality of teaching (Lalla, Frederic, & Ferrari, 2011; Carr & Hagel, 2008). Some previous 
studies have dealt with the issue of whether and to what extent the evaluation results 
truly reflect students’ attitudes. According to most authors, teacher rating proved to be a 
good indicator of teaching effectiveness (e.g., Beran & Violato, 2005; Nargundkar & 
Shrikhande, 2012; Wiers-Jenssen, Stensaker, & Grogaard, 2003). Beran and Violato 
(2005) also found that teacher rating is, to a lesser extent, biased by some factors that 
are not related to the teachers themselves, such as students’ grade expectations, 
attendance, and types of courses being evaluated. For example, teachers of elective 
courses have always been rated higher than teachers of compulsory courses, as shown 
by all findings to date. In previous research on teacher-related factors that affect ratings, 
interaction, teacher feedback, and communication appear most often (e.g., Kuo, Walker, 
Belland, & Schroder, 2013; Loveland, 2007; Rothman, Romeo, Brennan, & Mitchell, 
2011). Some other factors having the highest impact on learner satisfaction in online 
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education are: teacher knowledge and facilitation, course structure (Eom et al., 2006), 
appropriateness of readings and tasks, technological tools, course organisation, clarity 
of outcomes and assignments, and content format (Rothman et al., 2011). According to 
the community of inquiry framework (Garrison, Anderson & Archer, 2000, 2010), a 
deep and meaningful online learning experience happens through the evolution of three 
interconnected elements: cognitive, social, and teaching presence. In an educational 
environment, teaching presence is normally the main responsibility of the teacher. It 
includes selection, organization, and presentation of the course content, and the design 
and development of learning activities. Although the teacher’s personality is known to 
be a crucial factor in establishing a successful online learning atmosphere (Northcote, 
2010), the most apparent teachers’ characteristics, such as their qualification 
experience, do not always appear to be associated with teaching quality (Hanushek & 
Rivkin, 2010). Still, expressing the personality beyond the provision of mere resources is 
important because just the interaction is not enough to achieve a sense of teacher 
presence in online learning contexts (Garrison,Cleveland-Innes, & Fung, 2010). 

Our research was aimed at defining the characteristics and activities that will influence 
the rating of teachers on distance learning study programs. The data from the 
evaluation questionnaire, conducted regularly for the quality control of teaching and 
study processes at Singidunum University, have been used for the research. Singidunum 
University is the largest private university in Serbia, with more than 10,000 students 
enrolled in degree programs in the fields of finance, banking, accounting, marketing and 
trade, tourism and hotel management, engineering management, computer science, and 
electrical and computer engineering. Six percent (about 600 students) of this number 
attend distance learning programs from all the abovementioned fields except computer 
science and electrical and computer engineering. The research results are expected to 
benefit teaching staff as well as the management of higher education institutions, which 
aims to improve the quality of teaching delivery and increase satisfaction of students on 
distance learning programs. 


Method 


The research covered all four years of various study programs. The data were collected 
from both the University information system and the Moodle platform, then cross- 
analysed by using the SPSS statistical software. 

Numerous parameters were analysed as potential correlates of teacher rating, some of 
them being the number of teachers engaged per course, number of courses assigned to a 
particular teacher, gender and age of a teacher, total number of the available resources, 
and students’ activities, and so forth. Available resources and students’ activities were 
considered collectively as well as individually against the following categories: files, 
quizzes, forums, lessons, assignments, labels, and dictionaries. The Pearson correlation 
was used to measure the ratio between the two variables. Linear regression was used to 
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define the relationships between the variables, while t test was used to test the 
differences in the average values. The method of Spearman’s rank-order correlation was 
used to determine the association between variables that are not normally distributed. 
In order to test for association between two non-normally distributed variables while 
simultaneously controlling for the effect of already known confounders, the method of 
partial Spearman’s rank correlation was used. 

In order to create a comparison with the traditional studies, this research was conducted 
on a sample of 71 teachers, simultaneously engaged with both traditional classroom 
programs and distance learning studies. The total number of students that assessed the 
work of teachers in traditional classroom programs was 7,156, whereas 528 students 
assessed the work of teachers on distance learning studies. Each student rated the 
teachers only during the term s/he was currently in attendance, awarding grades from 1 
to 5, 1 being the lowest, and 5 the highest. The total number of polls completed was 
36,151 on traditional studies and 2,675 on distance learning study programs. The 
evaluation questionnaire comprised five questions: 

• Did the teacher provide appropriate learning resources that were 
prepared in a clear and understandable fashion? 

• Is communication with the teacher appropriate, relevant and timely? 

• Has the teacher provided you with the necessary theoretical and 
practical knowledge? 

• Is the teacher encouraging critical thinking? 

• Is the teacher encouraging inclusion of students into the study process? 

Students were also able to leave a comment at the end of the poll, related to the 
teacher’s performance. As the mean evaluation score for the teachers involved more 
than one course, we used the grand mean they received for all the courses they were 
involved in. At the end of this paper, we also summarised students’ remarks, which were 
usually given descriptively in the polls. 


Results and Discussion 


Because of the differences between online and traditional courses, some researchers 
have examined if and to what extent the same student ratings can be used in online 
courses. For example, students of traditional studies establish more frequent personal 
contact with their teachers, as opposed to the students of distance learning studies. Due 
to the lack of personal contact and empathy, the rating of distance learning teachers is 
often more reflective of the quality of the course content than the quality of the overall 
work of the teacher. This can provide misleading results, particularly with the courses 
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involving more than one teacher. For these reasons, we divided the courses into 
modules, with the name of the author clearly given for each online resource. A large 
number of recorded lectures were also uploaded; the communication on each of the 
courses was personalised so that the students could easily master the materials and have 
a clear idea of whom they were communicating with. 

Figure 1 represents the distribution of scores that the students awarded to the teachers 
on traditional studies and distance learning studies. The average values, standard 
deviations and the sample size are shown in this graph. The obtained results show that 
there is a positive association between the score awarded to the teacher on traditional 
studies and the score awarded to the teacher on distance learning studies (K69) = 0.575, 
p < 0.01). 
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Average evaluation score from students participating in 

the traditional classroom program 

Figure 1. Scatter plot of mean evaluation scores received from students participating in 
the two programs with “rug” density plots on margins. 
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The linear model has the following form: 

Average evaluation score from students participating in the online distance learning 
program = 0.880 + 0.756 x average evaluation score from students participating in the 
traditional classroom program 

The reliability analysis has also confirmed that the evaluation scale items are reasonably 
interrelated (5 items; Cronbach's a = 0.82). However, Sijtsma (2009) reminds us that a 
high value of Cronbach’s alpha does not provide evidence of unidimensionality. 
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Although distance learning involves reduced learner-tutor interaction due to 
asynchronous computer-mediated (ACM) conferences, it is obvious that certain teachers 
are able to meet the requirements, regardless of format or mode of delivery. This points 
to a very important conclusion: "good teaching" is just "good teaching" regardless of the 
medium. Comparing traditional and online courses, Beattie, Spooner, Jordan, 
Algozzine, and Spooner (2002) found similar results across course, teacher, and general 
ratings, regardless of the mode of delivery. Also, in the IDEA survey 2002-2008 by 
Benton, Webster, Gross, and Pallett (2010), students’ ratings of teachers, both of online 
and traditional courses, as well as ratings of the courses themselves, were all very much 
alike. They found some minor differences (e.g., that teachers of online courses were 
perceived as using educational technology more effectively), which is expected given the 
specifics of the mode of educational material delivery. 

According to Alonso Diaz and Blazquez Entonado (2009), teachers’ roles in online 
education are not very different from those of traditional courses. In both teaching 
modes teachers still have to deal with facilitating the teaching/learning process, 
combining activities with theoretical content, and encouraging student interaction. 
Kelly, Ponton, and Rovai (2007) also compared students’ ratings of overall instructor 
performance and overall satisfaction of online and traditional courses. They found no 
significant difference in students’ ratings between online and traditional courses. The 
authors propose that teachers of online courses should be fair and unbiased, and 
enthusiastic and helpful, and they should show real interest in student progress and 
needs. 

Based on the data from Table 1, which shows that the average teacher rating on 
traditional studies is higher than on distance learning studies by 0.13, and the results of 
paired samples of t test shown in Table 2, it can be concluded that such a difference is 
statistically significant (f(70) = -2.68, p < 0.01). The effect size for this analysis (d = 
0.31) is moderate according to the general conventions (Cohen, 1992). One of the 
reasons for this difference may be the lack of personal contact and empathy. It is well 
known that communication is experienced differently through different forms of 
communication media. It is much easier to develop personal understanding and 
empathy in face-to-face communication than in online communication, which cannot 
convey delicate social cues beyond the literal meaning of the words from the text 
(Lewicki, Barry, & Saunders, 2010). 


Table 1 


The Mean Evaluation Scores Received from Students Participating in the Online 
Distance Learning Program and the Traditional Classroom Program 


Distance learning program 
Traditional program 


Mean N Std. deviation 

4.00 71 0.47 

4-13 7i 0-366 


Std. error mean 

0.055 

0.042 
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Table 2 

Paired T-Testfor Difference in Means between Evaluation Scores Received from 
Students Participating in the Online Distance Learning Program and the Traditional 
Classroom Program 


Paired differences 








Std. 

error 

mean 

95% confidence 



Sig. 

(2-tailed) 

Mean 

Std. 

deviation 

interval of the 
difference 

Lower Upper 

t 

df 


-0.12 

0-39 

0.047 

-0.22 -0.03 

-2.68 

70 

0.009 


The second part of our research shifts the point of view from course-oriented to a rather 
more teacher-oriented one. The results showed that the number of teachers engaged per 
course is negatively associated with their average score. This association is significant, 
even when it is controlled for the score on traditional studies, which can be seen in 
Table 3 (rs(i73) = -0.196, p < 0.01). We assume that this is because of the lack of 
coordination among different teachers, due to their different ages, personalities, 
backgrounds, communication styles, and preferences. Another problem in these cases 
may be an excess of learning material and too many activities on the online platform, 
because each teacher supplies his/her own resources. 

Table 3 


Correlation between Mean Evaluation Scores for Specific Online Courses and the 
Number of Teaching Staff Involved in these Courses 



Average evaluation score from students 
participating in the online distance 
learning program 

Number of 
teaching staff 
involved 

Correlation 

1.00 

-0.19 

Significance (2-tailed) 

- 

0.009 

df 

0 

173 

Correlation 

-0.19 

1.00 

Significance (2-tailed) 

0.009 

- 

df 

173 

0 


The results also showed that the age difference of the teachers engaged in a particular 
course reduced the overall average score (rs(i72) = -0.177, p < 0.05). The age difference 
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is measured by the coefficient of variation (standard deviation of age divided by the 
average age of the teaching staff). 

Table 4 

Correlation between Mean Evaluation Scores for Specific Online Courses and 
Disparity in Age among Teaching Staff Involved in these Courses 



Average evaluation score from 
students participating in the online 
distance learning program 

Coefficient of variation 
of teaching staff age 

Correlation 

1.00 

-0.18 

Significance (2-tailed) 

- 

0.02 

df 

0 

172 

Correlation 

-0.18 

1.00 

Significance (2-tailed) 

0.02 

- 

df 

172 

0 


It is interesting that no statistically significant correlation between other variables has 
been perceived. The average teacher rating on distance learning studies, for example, is 
neither related to the number of uploaded teaching resources nor to the number of 
teachers’ activities on the online platform. These results lead to a conclusion that the 
mere number of the uploaded resources does not indicate their quality, and has no 
influence on student satisfaction. Estelami (2012), who tested student satisfaction in 
both hybrid-online courses (a combination of online and traditional classroom teaching) 
and purely online courses, reached similar conclusions. He defined several key factors 
that influence student satisfaction and that depend on teachers either directly or 
indirectly: course content, student-teacher communication, the use of effective learning 
tools, and the teacher him/herself. Considering the importance of learning tools, he 
concluded that the usefulness of textbooks and other support material, and not the 
number of resources, greatly influence students' general feelings and attitudes. Carr and 
Hagel (2008) also confirm that good quality teaching resources and higher levels of 
online activity are associated with higher satisfaction levels. 

Benton, Cashin, and Kansas’s (2012) review of literature and empirical studies revealed 
that the teacher’s personality does not influence student evaluation of teachers, and 
neither does the teacher’s race, gender, age, or research productivity. Paechter, Maier, 
and Macher (2010) investigated the factors that students regard as important for their 
satisfaction and performance. The teacher’s expertise in online learning and his/her 
level of support offered seem to be the best predictors for student satisfaction and 
learning achievements. 
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The fact that no correlation between the average rating and the age of the teacher was 
observed also speaks in favor of older generations of teachers being capable of adjusting 
successfully to the new teaching trends. Table 5 shows the average values and standard 
deviations of scores on distance learning study programs for two age groups of teachers, 
while Table 6 shows the results of t test, both of which confirm that difference in the 
average score does not depend on the teacher’s age. 

Table 5 

Mean Evaluation Scores Received from Students Participating in the Online Distance 
Learning Program, Categorised by Teaching Staff Age Groups 


The age of a teacher 

N 

Mean 

Std.deviation 

Std.error 

mean 

< 40 years 

36 

4.00 

0-43 

0.07 

>=40 years 

35 

4.01 

0.51 

0.09 


Table 6 

T-Testfor Difference in Mean Evaluation Scores between Teaching Staff Age Groups, 
Received from Students Participating in the Online Distance Learning Program 


Levene's test f-test for equality of means 

for equality of 

variances 


F 

Sig. 

t 

df 

Sig. (2- 
tailed) 

Mean 

diff. 

Std. error 
diff. 

95% confidence 
interval of the 
difference 

Lower Upper 

1.03 

0.31 

-0.03 

-0.03 

69.00 

66.62 

0.97 

0.97 

t 1 

0 0 
b b 
0 0 

0.11 

0.11 

-0.23 0.22 
-0.23 0.22 


It was not observed that the teachers engaged in more than one course had an average 
score lower than the mean value, which indicates that the workload of teachers, as long 
as it is within acceptable limits, is not a factor that affects the quality of work. 

Finally, we have summarised the most common objections to the work of teachers, 
which the students presented descriptively in the evaluation questionnaire. We believe 
that these remarks may help to understand the factors that affect students' satisfaction, 
and may also help to improve the teaching quality in distance learning study programs. 
As Kelly, Ponton, and Rovai (2007) posit, if there are differences between quantitative 
ratings of online and traditional courses, analysis of qualitative comments can provide 
us greater insight into those differences. 
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First of all, some teachers delay responses to students’ messages without apparent 
reason, and communication is sometimes scant, without necessary details. 
Communication with students on distance learning programs requires special attention, 
due to the fact that this type of study involves the increasing use of asynchronous video 
communication, as well as a lack of personal contact, which is essential for the 
development of trust (Dennen, Aubteen Darabi, & Smith, 2007). According to Wade, 
Cameron, Morgan, and Williams (2011), distance education students desire 
relationships with group members more than their colleagues enrolled in traditional 
study programs, and we can add that the same applies to their relationships with 
teachers. Furthermore, it has been established that facilitating students’ participation is 
one of the essential pedagogical competencies specific for e-learning practices (Munoz 
Carril, Gonzalez Sanmamed, & Hernandez Salles, 2013). 

Second, students find that some teachers do not upload enough learning and testing 
resources suitable for distance learning. The Moodle platform, which is nowadays used 
in many countries, offers a wide variety of content, applications, and forms of 
communication available in the e-learning environment, with a detailed description of 
options (Akhmetova, Vorontsova, & Morozova, 2013). Therefore, distance learning can 
rely on many technologies and features such as multimedia, streaming audio and video, 
web conferencing, instant messaging, peer-to-peer file sharing, and so on(McGreal & 
Elliott, 2008). However, apart from video tutorials and assessment tests with automatic 
feedback, such possibilities are rarely used, fully due to the omission in work of the 
teaching staff. 

The third objection, which is frequently encountered, is that most teachers set deadlines 
for the preparation of tests, mid-terms and finals, which do not fit all the students. The 
cause of the problem probably lies in the fact that the flexibility of online learning is too 
often taken for granted, and students interpret it as doing online assignments whenever 
it suits them. Although this interpretation is true of most online courses, sometimes 
students can have problems resulting from regular absence from online classes. 
Learners need to rely on stable terms and tasks, so that they can plan for their absence 
in advance. On the other hand, it is essential that the teachers provide more support to 
the students when they (the students) underestimate the time and effort required in 
online learning, since the lack of support is shown to be one of the main reasons for 
dropping out of university (Conrad, 2009). 

In the end, we should consider certain constructive remarks given by some authors with 
respect to teacher evaluations. Wiers-Jensen et al. (2003) state that evaluating student 
satisfaction remains controversial due to the contextual factors that can influence 
students’ perception of teaching quality. Several other authors (e.g., Becker & Watts, 
1999; D’Apollonia & Abrami, 1997) also found that students’ ratings are often influenced 
by teacher characteristics that have nothing to do with effectiveness, such as popularity 
or grading style. Still, many universities use student evaluation of teachers as the main 
factor in faculty promotion and salary, which raises a number of issues (e.g., Olivares, 
2003), including the validity (whether the results can accurately predict student 
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learning) and external biases (whether some other unrelated factors could influence 
student opinion) of such evaluations. It can also induce teachers to manipulate their 
grading policies, in order to boost their evaluations, which can ultimately lead to 
deterioration of education quality (Johnson, 2002). Griffin, Hilton, Plummer, and 
Barret (2014) analysed the grade point averages (GPAs) and teacher ratings over 2,073 
courses at a large private university. They found a moderate correlation between GPAs 
and teacher ratings, although this overall correlation did not hold true for individual 
teachers and courses. Although student ratings are useful in assessing the quality of 
teaching and of overall courses, they are not sufficient and therefore should not be used 
as a sole factor in determining teacher salary and promotion. 


Conclusions 


In this paper we analysed the main teacher-related factors that affect student evaluation 
of teachers on distance learning programs. The results show only a small reduction in 
ratings on DL studies, compared to traditional studies. The reason for this may be the 
lack of personal contact between students and teachers. We also found that the number 
of teachers engaged per course, as well as their age difference, lowered the average 
rating of the teachers involved in the course. On the other hand, no correlation was 
found between teacher ratings and teachers’ activities on the online platform, or the 
number of resources they uploaded. It shows that students appreciate more the quality 
and suitability than the quantity of learning materials. In order to improve distance 
learning education, teachers should devote special attention to communicating with 
students. When it comes to learning materials, teachers should use the advantages of 
the Moodle platform and give priority to the resources adapted to online learning, such 
as online tutorials and feedback on assignments. 
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